linux commands for hadoop administration

simply enters the command man and the name of the command shell will list a complete description.Standard usage:Name of the man commandTo query the description page of the LS command, enter the command: Man lsTo flip through the manual page, you can use the page UP and PAGE DOWN keys, or use the spacebar to turn back one page and use B to flip forward. To exit the manual page, enter the command Q. To search for keywords in the manuals page, enter a command/and keyword or phrase to search for, a

auto;height:auto; "/>DF View disk size df-h with unit display disk information650) this.width=650; "src=" http://p1.pstatp.com/large/1914000389081e28353a "alt=" Linux Learning ~linux System Administration Command "style=" border:0px;vertical-align:middle;margin:0px auto;height:auto; "/>PS ps-ef|grep Java query for Java process PID650) this.width=650; "src=" http

includes a file system of size 0 blocks -B,--block-size= size block with specified -H,--human-readable file system size in an easy-to-understand format (e.g. 1K 234M 2G) -H,--si similar to-H, but takes 1000 of the time instead of 1024 -I,--inodes displays inode information instead of block usage -K is--block-size=1k -L,--local only displays the native file system --no-sync No sync action (default) before getting usage data -P,--portability using POSIX output format --sync syn

example: useradd-d/home/bird bird2 ---bird2 is the account name,/home/Bird is the main directory of Bird2 -the general user of the system's main directory is generally built in/home/this pathpasswd Set User password -- in Linux, Superuser can use the passwd command to set or modify the user's password for a normal user. Users can also use the command directly to modify their own passwords without having to use the user name after the command. --pa

83Linux and 82Linux swap/Types of file systems supported by Linux d Delete the system partitionW Save exit and write to partition tableQ Do not save exit6. After reboot [best], create file system "similar to the format of Windows"Note: Any partition, if you want to save data, must have a way to manage the data, this way, called as a file system, such as Windows is ntfs[support disk quotas, file compression] or fat32;

also useful when creating a RAM disk.Module classLsmodDegree of importance: MediumLists all installed kernel modules.InsmodDegree of importance: MediumForce the installation of a kernel module (instead of using modprobe if possible) must be called as root.RmmodDegree of importance: MediumForces the uninstallation of a kernel module. Must be called as root.ModprobeDegree of importance: MediumThe module loader, in general, is automatically called in the startup script. Must be called as root.Depm

: Modify which NIC to follow which, only one can be omitted) x.x.x.x/y (M.M.M.M)//modify IP information; x:ip address, Y: Mask length, M: mask. such as =225.225.255.0Shutdown-h now or Poweroff or init 0//shutdown operation.Shutdown-r now or reboot or init 6//restart operation.Shutdown-r +15 ' The system would be rebboted!! '//Send a notification to each user who is currently logged on to the Linux host ' the system would be rebboted ', and reboot afte

1.1)vim/etc/udev/rules.d/ --persistent-Net.rulesVI/etc/sysconfig/network-scripts/ifcfg-Eth0type=Ethernetuuid=57d4c2c9-9e9c-48f8-a654-8e5bdbadafb8onboot=yesnm_controlled=YesBootproto = staticDefroute=Yesipv4_failure_fatal=Yesipv6init=NoNAME="System eth0"HWADDR=xx: 0c: in: -: E6:ecipaddr =172.16.53.100PREFIX= -gateway=172.16.53.2Last_connect=1415175123dns1=172.16.53.2The virtual machine's network card is using the virtual network cardSave Exit X or Wq2)Vi/etc/sysconfig/networkNetworking=yesHostnam

commands in the Hadoop process:1. View the process Linux under View process: $ PS Linux View Java process: $ jps 2. View Hadoop-related commands $ hadoop (using

success:mysql-h172.16.77.15-uroot-p123 mysql-h host address-u user name-P user PasswordView Character SetsShow variables like '%char% ';To Modify a character set:VI/ETC/MY.CNF add Default-character-set=utf8 under [client]create sudo without password loginTo set the Aboutyun user with no password sudo permissions: Chmode u+w/etc/sudoersaboutyun all= (root) nopasswd:allchmod u-w/etc/sudoers test: sudo ifconfigUbuntu View Service List codesudo service--status-allsudo initctl listTo view the file s

1.hadoopView the directory on HDFs: hadoop fs-ls/ Create a directory on HDFs: -mkdir/jiatest upload the file to HDFs Specify directory: -put test.txt /Jiatest upload jar package to Hadoop run: hadoop jar maven_test-1.0-snapshot.jar org.jiahong.test.WordCount/ jiatest/jiatest/Output View result: -cat/jiatest/output/part-r-000002.linuxU

Modify permissions for a file or directory Chown chown [ options ] User [ . Group ] file/dir Modify the owner of a file Chgrp Chgrp [-R] Group name Dir/file Modify the owning group of a file Iv. Systems and Networks Option name Meaning passwd xxx Change Password Df-ah View disk space Ps-ef |grep View process Kill-9 Kill the process

Ps-ef--View the processes in the systemPs-ef | More--View the process in the system, can be a page by Page viewPs-ef | grep FIREWALLD--View firewall processIn Linux, processes are generated by the parent process, and there is only one process that does not have a parent process.is the process with ID 0--pid: Process PPID: Parent processWhich FIREWALLD--see what executable programs firewalld haveKill 797--stop FIREWALLD This process, FIREWALLD process

(1) Process management1. Determine the server health status2. View the processA. Viewing all processes in the system: PSExample: PS aux (using BSD OS command format)A show all foreground processesX Show all background processesU shows which user generatedPs-le (using the Linux Standard command format)-L display more detailed information-E Show All processesResult descriptionVSZ: Occupies virtual memory size kbRSS: Occupies actual physical memory size

[Linux] [Hadoop] Run hadoop and linuxhadoop The preceding installation process is to be supplemented. After hadoop installation is complete, run the relevant commands to run hadoop. Run the following command to start all servic

, no permission is- The fourth character represents the owner execution permission, if the permission is x, no permission is- Fifth character represents the owner of the same group of user Read permission, if the permission is R, No permission is- The sixth character is the owner of the same group of user write permissions, if the permission is W, no permission is- The seventh character indicates that the owner is the same as the group user execution permissions, if the permission is x, no permi

Environment and objectives: -System: VMWare/Ubuntu 12.04 -Hadoop version: 0.20.2 -My node configuration (Fully Distributed cluster) Master (job tracker) 192.168.221.130 H1 Slave (Task tracker/data node) 192.168.221.141 H2 Slave (Task tracker/data node) 192.168.221.142 H3 -User: Hadoop_admin -Target: Hadoop, http: // localhost: 50

follows:A, enter the Conf folder to modify the following file.Add the following to the hadoop-env.sh:Export Java_home = (JAVA installation directory)The contents of the Core-site.xml file are modified to the following:The contents of the Hdfs-site.xml file are modified to the following: (Replication default is 3, if not modified, datanode less than three will be error)The contents of the Mapred-site.xml file are modified to the following:B. Format th

A. Common Hadoop commands1. The FS command for Hadoop#查看hadoop所有的fs命令Hadoop FS#上传文件 (both put and copyfromlocal are upload commands)Hadoop fs-put jdk-7u55-linux-i586.tar.gz hdfs://hucc0

A. Common Hadoop commands1. The FS command for Hadoop#查看hadoop所有的fs命令Hadoop FS#上传文件 (both put and copyfromlocal are upload commands)Hadoop fs-put jdk-7u55-linux-i586.tar.gz hdfs://hucc0